Virtual microscope-device and method

ABSTRACT

The invention relates to a virtual microscope system consisting of digital image data ( 31 ) of a preparation which is scanned image capture system with predefined magnification and a predefined detail or otherwise imported; a server system ( 2 ) with software ( 231 ) for generating a virtual profile ( 212 ) of the preparation from the digital image data ( 31 ) of the preparation which is provided by the image capture system or otherwise imported; software ( 232 ) for image processing of the virtual profile ( 212 ) and a storage device ( 21 ) which is used to store the virtual profile ( 212 ) in a case data bank ( 222 ); and at least one client system ( 1 ) with application software ( 11 ) for the representation of data of a selective virtual profile ( 212 ) from the case data bank ( 222 ) which can be selected by a user ( 13 ). In order to fully replace a conventional microscope with regard to the image representation thereof, the image data ( 31 ) of the preparation provided by the image capture system and the virtual profile ( 212 ) generated by the server is an image of the preparation with the highest available or desired degree of magnification and represents the highest available or desired profile of the preparation, particularly a general view. Functions such as the transmission of image data in real time, continuously variable zooming, the generation of multilayer images, a marking plane, processing of said images by means of image processing, case data management and protocolling et al. can be provided for any specific number of on-line clients.

The present invention relates to a virtual microscope system includingdigital image data of a preparation which is scanned at a definablemagnification and in a definable section by an image recording system oris otherwise imported; a server system having software for generating avirtual profile of the preparation from the digital image data of thepreparation provided by the image recording system or otherwiseimported, software for image processing of the virtual profile, and astorage device for storing the virtual profile in a case database; andat least one client system having application software for representingdata of a selectable virtual profile in the case database selectable bya user. The present invention also relates to a method for processingdigital microscope data as recited in claim 36.

To assess objects, such as histological preparations, processorcircuits, or other medical preparations or technical components withsignificant magnification, light-optical microscopes are typically used.Light-optical microscopes may be further classified according to thetype of preparation illumination (reflected light microscope ortransmitted light microscope), wavelength of the illumination light(simple light-optical microscope or fluorescent microscope), oscillationdirection of the illumination light (polarization or interferencecontrast microscope), as well as the phase position of the detectedlight (phase contrast microscope or dark field microscope). Alllight-optical microscopes are limited with respect to resolution as aresult of the wavelength of the light and therefore have a minimallydistinguishable point distance of approx. 0.2 micrometers (0.2×10⁻⁶ m).Under consideration of the minimum point distance discernible by thehuman eye of approx. 0.2 millimeter, this means a maximum possiblemagnification of approximately 1,000 times. Higher resolution isachieved using electron microscopes. In this case, the preparation isbombarded with the significantly shorter wave electrons instead of withlight wave, and their deflection as they pass through the preparation isdetected. As a result, a resolution having a point distance of approx. 2angstroms (2×10⁻¹⁰ m) is able to be achieved in theory. However, as aresult of the limited achievable lens quality, a visible enlargement ofonly approx. 300,000 times is actually possible. Electron microscopesdiffer according to the type of beam path (transmission microscope orscanning electron microscope).

All of these devices are subject to the following limitations:

-   -   Only one section may be viewed at one time.    -   Only a few specific lens-dependent magnification levels are        available.    -   Preparations may only be viewed one at a time.    -   It is not possible to make marks on the local microscopic level.    -   Content-related presentation is not possible.    -   Only one local viewer is possible at a time.    -   A preparation is only available once and is not able to be        duplicated.

The ability to comment on a case is also very limited. Notes or commentsregarding the image are typically made in writing or are spoken into adictation device. It is not possible to link notes to concrete sites onthe preparation in a clear manner without auxiliary devices (forexample, scanning table and software). The user must generally make anote of the sites of interest. It is not possible to send a case indigital form or to assess an object at a spatial distance from themicroscope without special auxiliary devices (e.g. a TV camera).

Various auxiliary components addressing individual technical limitationsof a microscope have be developed in the past to improve thefunctionality of such microscopes. As a result, a “multi-view device”allows a plurality of persons to view a preparation simultaneously via abeam-splitting lens to be attached to the microscope. The narrow fieldof view is able to be expanded by a special ocular lens (large-fieldlens). Video cameras including a monitor allow a plurality of persons toview the preparation at the same time but with very limited quality. Thegeneration of digital images of an individual section and theirduplication have also been achieved using a video camera and a computerconnection. Finally, a number of telepathology systems allowing the livetransmission of the microscope image to a remote user as well as remotecontrol of the microscope by this remote user have been available forseveral years.

In the beginning, digital methods for supporting microscopes were alsocreated. As a result, the creation of digitalized microscope images,known as virtual profiles (VP), via computer-controlled microscopes isknown from WO 98/39728, for example. In this context an overview imagecomprised of a plurality of partial images and having low resolution isrecorded and the partial images are linked to a common, coordinate-baseddata structure and stored. The user may subsequently select regions ofinterest in which the data is measured at a certain higher resolution ormagnification level and are also stored in a coordinate-based manner.The data structure described in this document does not provide for theprovision of a total image of the preparation at a higher resolution. Itis also not possible for the user to generate multi-layer images, to usea drawing plane, or to perform continuously variable zooming.

It is known from WO 01/54052 to store the data of a virtual profile on aserver and to visualize this on the client side via a Web browser in theform of an overview image at a low resolution and callable partialimages at a higher resolution, switching back and forth between thedifferent resolutions being provided. A total image at the highestresolution is also not provided in this instance. Moreover, it is notpossible to use multi-layer images and a drawing plane. There is also nocontinuously variable zooming or user data and case data management.

WO 99/47964 describes special techniques for scanning a preparation.

Finally sending the virtual profiles as a data structure via theInternet and intranet is known from WO 01/26541. Live discussion of acase by a plurality of users via the Internet is also known fromtelepathology.

In conclusion, a system which could completely replace a microscope(apart from image recording and scanning) is not yet available. Such asystem would be desirable since it would combine all advantages in onesolution, would be less expensive and more convenient, and would alsorender possible functions urgently desired by microscope users. Suchfunctions include, for example, the use of multi-layer images, the useof a marking plane, automatic case management for workflow automation,automatic image processing for supporting and accelerating the work of amicroscope user, as well as continuously variable zooming.

Thus, the present invention is based on the object of providing avirtual microscope which completely replaces all functions of aconventional mechanical light-optical or electron microscope, with theexception of the actual image recording function, and also providesseveral additional functions previously unknown in the mechanicalsolution. Furthermore, the virtual microscope is to overcome the knownlimitations of today's digital microscope and render possible real-timeremote transmission of image data in particular. A suitable method forprocessing digital microscope data is also to be provided.

The object is achieved by a virtual microscope having the features ofclaim 1 and a method as recited in claim 36. The virtual microscope ofthe present invention is characterized in that the digital image data ofthe preparation which is transmitted by the image recording system, e.g.a light-optical or electron microscope, or is otherwise imported (forexample from PACS) or the virtual profile on the server side generatedfrom this data is an image of the preparation at the highest availableor desired magnification and in the greatest available or desiredsection of the preparation, in particular a general view of thepreparation.

According to the present invention, the database therefore includes onlyone digital microscope image of each preparation at a single, i.e.maximum, magnification. In this context, high or low magnificationrefers to a representation on the user side at a high or low resolutionas is typical when representing digital images on monitors. In thismanner, a user is always able to access virtual profiles directly from aclient station without new microscopic measurements having to beperformed in the region of interest at the desired magnification, forexample.

The basic principle of the present invention is therefore the scanningof a whole preparation at the highest available or necessarymagnification to form a total image, its representation in an Internetor intranet-based or independent user interface, and the processing ofthe case using new digital tools in a client-server environment. Thecomplete object slide is digitalized at the highest magnification and isstored on a computer (server) as an electronic file (virtual profile).The image may come from a digital camera installed at a microscope(possibly remote-controllable), imported digital images, another imagerecording system, or a PACS (picture archiving and communicationssystem). The image data is preferably available in a compressed formsince very large data quantities are to be handled. The image may beloaded onto the system via Internet or intranet protocols (such asTCP/IP) or a fixed data carrier (such as a CD ROM). The virtualmicroscope system which visualizes specifically the desired preparationsection or also the entire preparation on a computer monitor of a clientsystem is used for viewing the virtual profiles (VS). Control operationsare performed via an interactive input medium (for example, a mouse,joystick, touch pad, etc.) by the user.

Assessing objects at significant magnification in digital form resultsin significant time-related and functional improvements. Once an objecthas been digitalized via a suitable image recording system (for example,using a conventional microscope having a scanning table and digitalcamera), the system of the present invention allows the image to beavailable to any desired user, even a plurality of simultaneous users,via an intranet or the Internet. The digital image may be superimposedby an additional layer (marking plane) on which comments or notes may bemade or sites of interest may be marked. In addition, the digital formrenders possible continuously variable zooming and shifting as wellas-various image processing methods.

According to a particularly advantageous embodiment of the presentinvention, the application software of the client system includes aviewer in which the general view of the preparation is represented as anoverview image having low resolution and/or at least one image regioncurrently selected by the user is represented as a detailed view athigher resolution. Preferably, the overview image which displays theentire preparation and preferably the region of the selected sectionmarked thereon is constantly represented in addition to therepresentation of the selected preparation section. A plurality ofsections may also be viewed simultaneously. The resolution may be freelyset by the user in the overview image as well as in all detailed views.

Furthermore, means via which exactly one image region of the selectedvirtual profile selected by the user from the case database of theserver system is currently requested and displayed on the client side inthe viewer are advantageously provided in the client system. The datamay optionally also be loaded to the client server system via mobiledata carriers (for example, CD ROM). The highly compressed image datagenerated by the software for generating virtual profiles and preferablyavailable in the enhanced compressed wavelet format (ECW, format ofEarth Resource Mapping) allows the quasi real-time transmission andrepresentation of the selected region. In this context, only the data ofthe image region of interest is transmitted in each case at thecompression level of the resolution of interest from the server systemto the client system, decompressed on the client side via decompressionsoftware, preferably in the form of an Earth Resource Mapping plug-insuitable for the compression format, and represented with the desiredsize in the viewer.

Since the total image of the preparation is available at the highestresolution in the form of the virtual profile, a continuously variabledigital zooming function allowing a detailed view within any site in thegeneral view of the virtual profile up to the highest magnification (orresolution) is achieved according to the present invention. For thispurpose, the user places, manually for example, a tracker provided inthe general view or the detailed view to specify the position andexpansion of the desired sub-region whereupon the client system requeststhe necessary image data from the server system and represents it as anew detailed view. In this context, the representation of a section orimage at a low magnification corresponds, as is typical in digitalzooming, to the representation of the image at a low resolution. As aresult, a preparation is able to be displayed in a continuously variablemanner at every magnification level. The zooming is performed in realtime so that the user is able to set the optimum magnification levelinteractively. The tracker may also be used for sampling the virtualprofile via manual shifting.

A particularly advantageous embodiment of the present invention showsthe possibility for using multi-layer images made up of a plurality ofsuperimposed total images of the same preparation, measured for exampleusing different colorings or recording techniques. Multi-layer imagesare known in principle. A multi-layer image in this context is an imagedata set including a plurality of images of the same object within onefile which are geometrically adjusted with respect to one another. Thisallows the VM user actual simultaneous viewing and as such direct andquick visual comparison of two or more images of an object as was notpreviously possible using currently available microscopes and digitalsolutions with respect to the total image of the preparation.Simultaneous visualization of images is achieved by dividing thedetailed view, quickly and simply switching the view, or also viasemi-transparent viewing of a plurality of preparations having thepossibility at the same time of adjusting the transparency. Thegeometric adjustment of the images (matching) with respect to oneanother is performed within an image processing module on the server.

In this context, it is preferably possible in particular to represent aplurality of detailed views at the same time, the detailed views beingrepresentable in particular at different magnifications and/or withdifferent coloring and/or at different locations and/or as differentzoom representations.

A further significant advantage of the present invention is that theimage or multi-layer image is superimposed by a marking plane on whichthe user is able to make image coordinate-based comments, markings,and/or the like. This allows marking of sites or regions of interest,entering of image coordinate-based texts, symbols, graphics, drawings,markings, hyperlinks, references to other data (e.g. audio data), anddrawing, even freehand, in the image, as well as the linkability ofmarked sites and additional preparation information as needed. Inaddition, the marking plane may include markings of image regions viewedduring image analysis and/or of an inspection path which represents apath through the preparation inspected by the user. The marking planemay be optionally displayed or may be invisible. It is also possible tostore the marking plane together with the virtual profile permanently inthe case database.

According to a preferred embodiment of the present invention, thesoftware for generating virtual profiles which is installed on theserver system includes an import function which allows the import ofimage data provided by the recording system in any image data format.The image data formats are then converted by the software into thecompressed image format (in particular in ECW format), a color depthbeing able to be variably defined. The image data of a virtual profileis stored in this form. In addition to the actual image data, thevirtual profiles stored in the case database include furtherinformation, in particular status data of the case, preparation andrecording parameters, and/or the like, this text data preferably beingavailable in a SQL structure (structure query language, a diverse queryformat for databases). The text data may be entered by an operator ofthe recording system and/or by a user on the client side via typicalinput means.

A further particularly advantageous embodiment of the present inventionprovides for server-side provision of software or a software packet forimage processing of virtual profiles which may be started by the serversystem automatically, according to the case type after generation of thevirtual profile, or be performed individually by a user, the imageprocessing including one or more of the steps:

-   -   Contrast improvement    -   Normalization of the brightness distribution    -   White balancing    -   Edge extraction    -   Matching to known structures for preselection of regions of        interest    -   Structure measurement and counting        and/or additional functions. According to further preferred        embodiments of the present invention, such image processing        functions may also in particular be simulation of illumination        situations, in particular simulation of at least one bright        field and/or dark field, and/or polarization and/or incident        light and/or transmitted light and/or phase contrast or the        like. Furthermore, the image processing function may be a        simulation of color situations, in particular generation of        mixed coloring as a combination of real colorings and/or color        change. The image processing steps may either be called up by a        user or automatically started according to the particular case        type. A list of image processing steps may be created for        relevant case types so that the system automatically starts        defined steps according to the current case type. Image        processing is used for automatically emphasizing structures        which support the user in subsequent analysis of the image or        direct the user in a targeted manner to relevant structures        (e.g. mitoses in tumor preparations).

A further advantageous embodiment of the present invention providesmeans for managing the case, image, and user data. In this context, inaddition to the case database in which all case data, image data, andstatus data of a case are stored in a structured manner (for example,using SQL), there is a user database in which access data, accessrights, and case allocations are stored. The server system on whichthese databases are stored ensures secure, redundant storing of the data(for example, via a RAID system). The careful configuration of these twodatabases first makes it possible to replace a conventional microscopecompletely with the virtual microscope system. This also includesdetailed protocolling of the workflow and the work of the user.

Another preferred embodiment of the present invention provides for theaccess control of users to the server system to include means forencoding/decoding and/or means for checking access authorization, inparticular as a digital signature. This allow secure authenticationduring user accessing.

The system of the present invention also includes a conference modebetween the client systems, a control change, in particular amaster-slave allocation, being possible in particular between the clientsystems and/or the client system and the server system. As a result,live discussions, in particular audio-visual live discussions areadvantageously possible between the client systems and/or the clientsystem and the server system.

The method of the present invention for processing digital microscopedata includes the following steps:

-   a. Scanning or otherwise importing digital image data of a    preparation at a highest available or desired magnification and in a    greatest available or desired section of the preparation by an image    recording system;-   b. Generating a virtual profile at the highest available or desired    magnification and in the greatest available or desired section of    the preparation from the image data of the preparation which is    provided by the image recording system or is otherwise imported via    a first software of a server system,-   c. Changing the virtual profile via image processing functions of a    second software of the server system,-   d. Storing the virtual profile in a case database of a storage    device, and-   e. Representing data of a selected virtual profile of the case    database selected by a user of at least one client system via an    application software.

Further advantageous embodiments of the present invention are the objectof the remaining dependent claims.

The present invention is subsequently described in greater detail on thebasis of the corresponding drawing. The figures show:

FIG. 1 schematically shows the data flow of the entire system of avirtual microscope;

FIG. 2 schematically shows the mode of operation of the server systemwithin the total system; and

FIG. 3 schematically shows the mode of operation of a client systemwithin the total system.

According to FIG. 1, the virtual microscope of the present invention isa client server-based system. It includes the two main componentsconsisting of at least one client system 1 and a server system 2, aclient 1 also being able to be installed together with server system 2on the same computer.

Server system 2 includes a storage system 21 (FIG. 2) on which allvirtual profiles 212 of a case are stored. Server 2 also provides serversoftware 23, in particular a software 231 for generating virtualprofiles 212 and a software 232 for image processing of virtual profiles212.

All computer stations communicating via the Internet or an intranet withserver system 2 and operated by users 13 (FIG. 3) of the system areincluded in client system 1. Any number of client systems 1 may beavailable and may access server 2 at the same time. For communicationand data transmission purposes, every client system 1 has an interface15 and server system 2 has an interface 25, which are preferably TCP/IPinterfaces. An application software 11, which is able to run on everyclient system 1, is required on the client side for requesting,representing, and assessing a virtual profile 212. According to therepresented example of server system 2, application software 11 isprovided in a Web browser 111. However, application software 11 mayoptionally also be provided as an independent program interface orwithin a user interface of another software in client system 1.

The following describes several individual components of the server andclient system in greater detail.

Server

Server system 2 is made up of storage system 21, which is a read onlymemory in particular and includes a database module 22, software 23, andinterface 25.

Database module 22 has the following tasks:

-   -   Storing the digital images recorded by an image recording system        in the form of virtual profiles 212 in a case database 222;    -   Operating server software 23 via which the images and data may        be viewed and modified;    -   Managing the image and text data in databases 22;    -   Controlling access via a user access database 221, which manages        registered users 13 and their access rights.

The image data is stored in a special image format which allows highcompression (preferably in ECW format). The described data and case data(212) are managed in a SQL-based manner. This facilitates the finding ofspecific cases on the basis of all recorded parameters.

Server system 2 also includes several software components 23. Theseinclude:

1. Application software 11, which may be transmitted to Web browser 111or as an independent program of a user 13. This application software 11is used as the user interface. It includes a first display window inwhich an overview image of virtual profile 212 is represented and viawhich navigation is possible in the virtual profile, another displaywindow which displays a currently selected image section, and aplurality of control tools used for navigating in the image, forprocessing the case data, and for managing the data. This applicationsoftware may alternatively also be loaded on the client side from afixed data carrier (for example, a CD ROM).

2. A program 231 for generating virtual profiles 212. The program isnecessary for formatting digital image data 31 of an image recordingsystem into a virtual profile 212, which is then stored in case database222. A virtual profile 212 is initially made up of image data,multi-layer data also being possible, and some data 32, such as casenumbers, recording date, and the like which is entered interactively byuser 13 or is imported via an interface from another database system(e.g. hospital information system or PACS). However, the data of such avirtual profile 212 increases as the case is processed since variouscomments, markers, and the like are subsequently added. Therefore,program 231 for generating virtual profiles 212 has the task ofreformatting the transmitted image data into the special image format ofa virtual profile 212, combining partial images to form a total image ifnecessary, or matching a plurality of image layers together such thatthey have a common, geometric, for example coordinate-related basis. Inaddition, the data provided by user 13 must be entered into the formatof virtual profile 212. The virtual profile generation proceeds asfollows:

-   -   Digital image data 31, which is provided by the image recording        system in raw, tiff, jpg, or another format, is first loaded;    -   Various image processing steps are applied in a defined sequence        according to the case type (preparation-dependent);    -   The color is adjusted to a defined color depth (e.g. 16 bit=2×5        bit+1×6 bit);    -   A reformatting operation to a compressed image format (e.g. ECW)        is subsequently performed;    -   If available, superimposition and geometric matching of        additional image layers are performed to form one multi-layer        image;    -   Additional information from interaction with user 13 is applied;    -   Case number, processor, status, case type are created;    -   All image and text data is stored as a virtual profile 212 in        case database 222.

3. A software 232 for image processing of virtual profiles 212. This mayinclude all methods of image processing which may be used for digitalimages to automatically emphasize structures which support user 13 insubsequent assessment of the image. Such methods include, for example:

-   -   Contrast improvement,    -   Normalization of the brightness distribution,    -   White balancing,    -   Edge extraction,    -   Matching to known structures for the preselection of regions of        interest,    -   Detecting, measuring, and counting objects and structures,    -   Various others.

The image processing steps may either be called up by a user 13 orstarted automatically according to the particular case type. A list ofimage processing steps may be created for every case type in use so thatthe system automatically starts defined steps according to the currentcase type.

Finally an interface 25, via which server 2 may be connected to clientsystems 1, is also part of server system 2. This is preferablyimplemented via TCP/IP.

Client

Application software 11 is controllable on user side 1 via a Web browser111 or runs as an independent user interface and is consequentlyindependent of the system. The viewer for large image files is providedon user-side Web browser 111 or the independent program interface. Theimage data is stored entirely on the server side. In this context onlythe image data of interest is transferred from the server system to theclient system and decompressed. The special format for the image dataensures high compression without visible quality loss with quick accessto individual regions within the image file. A high compression rate isparticularly important for the large size of the image data. The quickaccess to the image data allows zooming and sampling of the imagesalmost in real time. The software is accessed via a login, in order tocontrol access rights and protect the data. As a result, each user isgranted access only to certain data and has certain administrativerights for every data type (for example create, read, modify, delete).

Application program 11 of client system 1 may include a series of thefollowing functions, points 1 a through 1 g, 2 h, 3 l, 4 p, and 7 xthrough 7 z representing particularly preferred basic functions, points2 i and 2 j, 3 m and 3 n, 4 q and 4 r as well as 6 v representingadvantageous practical main functions, and points 2 k, 3 o, 4 s, 5 t,and 5 u, 6 w as well as 7 aa representing optional functions having apurely auxiliary function:

-   1. Image movement may be achieved using variable input devices, the    following functions being provided:    -   a. Continuously variable moving of the image (interactive);    -   b. Continuously variable zooming of the image (interactive);    -   c. Shifting of the image by a certain amount (semi-automatic);    -   d. Adjusting of a certain magnification level (semi-automatic);    -   e. Selecting of an image section via a selection rectangle in        the overview image (semi-automatic)    -   f. Selecting of an image section via a selection point in the        overview image (semi-automatic) and/or    -   g. Selecting of an image mark (see below) (automatic).-   2. Image documentation via a marking plane includes the options:    -   h. Inserting of image marks (interactive);    -   i. Inserting of text or symbols (interactive);    -   j. Inserting of graphics or freehand drawing (interactive)        and/or    -   k. Inserting of references or links (interactive).-   3. An intelligent search of images or image regions includes:    -   l. Search for markers (automatic),    -   m. Search for text content of the marking plane (automatic),    -   n. Search for case data (automatic) and/or    -   o. Search for image content (structures, textures, forms,        colors) (semi-automatic).-   4. In addition, the following measurement functions may be provided:    -   p. Automatic counting of structures (automatic),    -   q. Segment calculation including specification of the segment on        a real scale (automatic),    -   r. Surface calculation including specification of the surface on        a real scale (automatic) and/or    -   s. Classification of textures or colors (semi-automatic).-   5. Additional image processing functions may include:    -   t. Matching of a plurality of image layers (geometric adjustment        of images of the same object using different coloring or        recording technique) (automatic), and/or    -   u. Comparing of a plurality of image layers (subtraction image)        (automatic).-   6. Additional functions may include:    -   v. Snapshot (storing of partial images) (automatic) and/or    -   w. Track history (registering of viewing path) (automatic).-   7. The following may be provided as management functions:    -   x. Inputting of an assessment (interactive),    -   y. User management (semi-automatic),    -   z. Case data management in the database (semi-automatic) and/or    -   aa. Tracking of a case along the entire workflow (automatic).        User Interface

The user interface runs within a Web browser 111 or an independentprogram. The program window (viewer) within the browser or programwindow is preferably divided into six regions. These six regions are notrequired to always appear simultaneously. Rather, the user may activate,deactivate, and arrange the desired interface elements.

1. Menu Bar

The menu bar includes controls for all functions provided by the virtualmicroscope. In addition, the management of a case may be controlled viathe menu bar and all options and system attributes may be set andmodified.

2. Overview Image

The overview image initially displays the entire digital preparation(virtual profile). During the course of the assessment, it is used forselecting a new image view and for representing the currently displayedimage region. A tracker, i.e. a rectangle, the edges of which match theedges of the image in the image view, appears in the overview image forthis purpose. Using different functions, a new image view may beselected via the tracker. However, the overview image may also be usedto display a partial view as desired.

3. Control Elements

The control elements include different graphic control elements tofacilitate navigation in the image. The mouse is used here for control.Alternatively, other input devices (joystick, touch pad, etc.) or themenu bar may also be used to navigate.

4. Image View

The image view shows the current partial image selected via the overviewimage or in another manner. Based on this image, user 13 is able toassess a preparation. The overview image is navigable in different ways.This may occur via the mouse, the control elements, the overview image,or other input devices. In addition, other image layers or the markingplane may be displayed in the image view, optionally also as asemi-transparent representation. A context menu via which location-basedfunctions (for example the setting of a marker) may be activated isavailable via the mouse.

5. Status Line

The status line shows program information for the user, such as progressbars, program status, etc. in one line.

6. Image List

The image list is a table of small partial images which compose theessential components of the virtual profile or also a table of overviewimages of all virtual profiles of one case. It is possible to select animage and to switch to the image view via control elements.

Input Devices

Devices via which a user may interact with the PC or input data aredesignated as input devices. This input is necessary during userinteractions (use of control tools) or during data input. The possibleinput devices are presented in the following table, particularlypreferred basic functions being represented by points 1 through 4,advantageous practical main functions being represented by point 5 andoptional functions having a purely auxiliary function being representedby points 6 through 8:

-   1. Image recording system    -   System for digitalizing image data-   2. Keyboard    -   For data entry and control-   3. 5-button scroll mouse-   4. Interface (e.g. HL7 or DICOM interface)    -   For importing data from other database systems-   5. Joystick    -   Tool for precisely moving an image. Also used for selecting menu        points or activating buttons and similar control elements.        6. Microphone    -   For data input (speaking audible comments, in connection with        voice recognition software for inputting text data) and        controlling (direct command entry via voice recognition, e.g.        “Zoom out!”, menu and user interface control via voice        recognition, e.g. “Open file drive A!”).-   7. Dataglove (glove having motion sensors for direct control of a    mouse pointer/image on the screen). Control tool for moving images    on the screen, activating menus or buttons and similar control    elements or also for moving 3D objects.-   8. Gesture recognition (camera having software for recognizing    gestures and hand movements. Recognition of line of sight also    conceivable). Control tool for moving images on the screen via hand    motion or simply by looking, activating menus or buttons and similar    control elements on the screen by pointing or simply looking.

Additional usable, optional input devices are a trackball, touch screen,and/or touch pad.

Output Devices

Output devices are devices via which a user is able to view computerdata. This includes devices via which a remote third party is able toview the data. The possible output devices are summarized in thefollowing table, particularly preferred basic functions again beingrepresented by points 1 through 3, advantageous practical main functionsbeing represented by points 4 and 5, and optional functions having apurely auxiliary function being represented by point 6:

-   1. Monitor (also a touch screen)-   2. Modem or network card    -   For sending a case or parts thereof (e.g. individual images) to        remote users via fax, e-mail, or direct connection (TCP/IP)-   3. Interface (e.g. HL7 or DICOM interface)    -   For exporting data to other database systems-   4. Loudspeaker    -   For outputting spoken comments and for communicating live        discussions-   5. Printer-   6. Video projector.    List of Reference Numerals-   1 Client system    -   11 Application software        -   111 Web browser or independent software user interface    -   12 Storage system        -   121 Access data    -   13 User    -   14 Access software on external databases    -   15 TCP/IP interface-   2 Server system    -   21 Storage system        -   211 Access data        -   212 Virtual profile    -   22 Databases        -   221 User database        -   222 Case database    -   23 Server software        -   231 Software for generating virtual profiles        -   232 Software for image processing of virtual profiles    -   24 External software    -   25 TCP/IP interface-   3 Input data    -   31 Digital image data of a preparation        -   32 Case data of a preparation-   4 External databases

1. A virtual microscope having a. Digital image data (31) of apreparation which is scanned in by an image recording system at adefinable magnification and in a definable section or is otherwiseimported, b. A server system (2) having a software (231) for generatinga virtual profile (212) of the preparation from the digital image data(31) of the preparation which is provided by the image recording systemor is otherwise imported, a software (232) for image processing of thevirtual profile (212), and a storage device (21) for storing the virtualprofile (212) in a case database (222), and c. At least one clientsystem (1) having an application software (11) for representing data ofa selectable virtual profile (212) of the case database (222) selectableby a user (13), wherein the digital data (31) of the preparationprovided by the image recording system and the virtual profile (212)generated on the server side are an image of the preparation at thehighest available or desired magnification and in the greatest availableor desired section of the preparation.
 2. The virtual microscope asrecited in claim 1, wherein the greatest available or desired section ofthe preparation is a total view of the preparation.
 3. The virtualmicroscope system as recited in claim 1, wherein the software (231) forgenerating the virtual profiles (212) includes a function forreformatting the digital image data (31) provided by the image recordingsystem to a compressed image format, and the virtual profiles (212) areavailable in the compressed image format in the case database (222), inparticular in enhanced compressed wavelet format (ECW).
 4. The virtualmicroscope system as recited in claim 1, wherein the applicationsoftware (11) of the at least one client system (1) includes means viawhich exactly one image region of the selected virtual profile (212)selected by the user (13) is requested at a compression levelcorresponding to a selected resolution from the case database (222) ofthe server system (2) or is loaded from a mobile data carrier andrepresented.
 5. The virtual microscope system as recited in claim 1,wherein the application software (11) includes a viewer in which thetotal view of the preparation and/or the image region currently selectedby the user (13) can be represented as a detailed view.
 6. The virtualmicroscope system as recited in claim 1, wherein the applicationsoftware (11) includes a continuously variable digital zoom function viawhich a detailed view of any location within the total view of thevirtual profile (212) selected by the user can be represented at up tothe highest resolution, the zooming being implementable in acontinuously variable manner via interactive selection of the imageregion and determination of a representation size by the user (13). 7.The virtual microscope system as recited in claim 5, wherein theapplication software (11) includes a tracker which is represented in thetotal view of the preparation, clarifies the position and expansion ofthe represented detailed view, and is manually settable, shiftable, anddimensionable by the user (13) for selecting the image region and formanual sampling of the virtual profile (212), thereby selecting a newdetailed view.
 8. The virtual microscope system as recited in claim 1,wherein the digital image data (31) supplied by the recording system isavailable in any image data format and can be imported by the software(231) for generating the virtual profile (212), the color depth beingvariably definable.
 9. The virtual microscope system as recited in claim1, wherein the virtual profiles (212) stored in the case database (222)include, in addition to the image data, other data (32) able to be inputby a user (13) of the recording system and/or a client system (1), inparticular status data of the case, recording parameters, preparationdata, and/or the like, all data being available in a SQL structure inparticular.
 10. The virtual microscope system as recited in claim 1,wherein the server system (2) also includes a software (232) for imageprocessing of the virtual profiles (212).
 11. The virtual microscopesystem as recited in claim 10, wherein the software (232) for processingvirtual profiles (212) includes image processing functions, inparticular contrast improvement; normalization of the brightnessdistribution; white balancing; edge extraction; matching to knownstructures for the preselection of regions of interest; recognizing,measuring, and counting objects and structures, and/or other functions.12. The virtual microscope system as recited in claim 10, wherein asimulation of illumination situations, in particular a simulation of atleast one bright field and/or dark field, and/or polarization and/orincident light and/or phase contrast or the like, is/are implemented asimage processing functions.
 13. The virtual microscope system as recitedin claim 10, wherein a simulation of color situations, in particulargeneration of mixed colors as a combination of real colorings and/orcolor change is/are implemented as image processing functions.
 14. Thevirtual microscope system as recited in claim 10, wherein one or moreimage processing functions of the software (232) is automaticallyimplementable as a function of the case type or individually by a user(13) following creation of the virtual profile (212).
 15. The virtualmicroscope system as recited in claim 13, wherein the access controlincludes means for encoding/decoding and/or means for checking accessauthorization (digital signature).
 16. The virtual microscope system asrecited in claim 15, wherein a conference mode is possible between theclient systems (1).
 17. The virtual microscope system as recited inclaim 16, wherein a control change, in particular a master-slaveassignment, is possible between the client systems (1 and/or the clientsystem (1) and the server system (2).
 18. The virtual microscope systemas recited in claim 1, wherein the server system (2) includes means(211, 221) for controlling the access of users (13) to the server system(2).
 19. The virtual microscope system as recited in claim 18, whereinthe means (211, 221) for controlling access include a user database(221) including stored user data, access rights, and allocations tocases and the like as well as access data (211).
 20. The virtualmicroscope system as recited in claim 1, wherein the at least one clientsystem (1) includes a client interface (15) and the server system (2)includes a server interface (25), which are TCP/IP interfaces inparticular, and communication between the client system (1) and theserver system (2) is implementable via the interfaces (15, 25) andintranet or Internet, a plurality of client systems (1) also being ableto access the server system (2) at the same time.
 21. The virtualmicroscope as recited in claim 1, wherein a client system (1) isinstalled together with the server system (2) on a computer station. 22.The virtual microscope system as recited in claim 1, wherein theapplication software (11) is provided within a Web browser (111) of theclient system (1) by the server system (2).
 23. The virtual microscopesystem as recited in claim 1, wherein the application software (11) isprovided as an independent program interface in the client system (1) orwithin a program interface of another software in the client system (1).24. The virtual microscope system as recited in claim 1, wherein meansfor allocating data of virtual profiles (212) to users (13), casestatus, case data, and data of the virtual profiles (212) are providedin the server system (2).
 25. The virtual microscope system as recitedin claim 1, wherein the server system (2) and/or the at least one clientsystem (1) include(s) means for generating multi-layer images from aplurality of superimposed virtual profiles (212) of the samepreparation, measured using different colorings and/or recordingtechniques.
 26. The virtual microscope as recited in claim 25, whereinthe generation of multi-layer images as well as the combining of aplurality of partial images to form one total image is supported byautomatic geometric matching.
 27. The virtual microscope system asrecited in claim 25, wherein a plurality of layers can be represented atthe same time by dividing the detailed view, switching the detailedview, or using semi-transparent representation, the degree oftransparency being adjustable.
 28. The virtual microscope system asrecited in claim 27, wherein a plurality of detailed views can berepresented at the same time.
 29. The virtual microscope system asrecited in claim 28, wherein the detailed views can be displayed atdifferent magnifications and/or with different colorings and/or atdifferent locations and/or in different zoom representations.
 30. Thevirtual microscope system as recited in claim 1, wherein the serversystem (2) and/or the at least one client system (1) include(s) meansfor generating a marking plane superimposed on the image data of thevirtual profile (212), image coordinate-based comments, markings, and/orthe like being able to be made by the user (13) on this marking planeand integrated with the virtual profile (212) and stored in the casedatabase.
 31. The virtual microscope system as recited in claim 30,wherein the marking plane includes a marking of the image regions viewedby the user (13) during analysis of a virtual profile (212) and/or of aninspection path representing the path through the preparation inspectedby the user (13).
 32. The virtual microscope system as recited in claim1, wherein the application software (11) includes image processingfunctions, in particular contrast improvement, normalization of thebrightness distribution, white balancing, edge extraction, matching toknown structures for preselection of regions of interest, measuring andcounting of structures, and/or additional functions.
 33. The virtualmicroscope system as recited in claim 1, wherein a virtual profile (212)includes retrievable tracking data including the processes andinformation relating to the virtual profile (212) and continuouslylogged along the entire workflow, in particular the current status, user(13), process, and the like.
 34. The virtual microscope system asrecited in claim 1, wherein the digital image data (31) of thepreparation provided by the image recording system and the virtualprofile (212) generated on the server side are an image of thepreparation made up of sub-regions.
 35. The virtual microscope system asrecited in claim 1, wherein the image sources are digital profiles,imported digital images, PACS (picture archiving and communicationsystem) images, live images (remote-controllable microscope withcamera), or the like.
 36. A method for processing digital microscopedata, a. Digital image data (31) of a preparation being scanned at adefinable magnification and in a definable section by an image recordingsystem or being otherwise imported, b. A virtual profile (212) of thepreparation being generated via a first software (231) of a serversystem (2) from the digital image data (31) which is provided by theimage recording system or is otherwise imported, c. The virtual profile(212) being modified using a second software (232) and image processingfunctions, d. The virtual profile (212) being stored in a case database(222) of a storage device (21), e. Data of a selected virtual profile(212) of the case database (222) selected by a user (13) being displayedvia an application software (11) of the at least one client system (1),wherein the digital image data (31) of the preparation is scanned by theimage recording system at the highest available or desired magnificationand in the greatest available or desired section of the preparation andprovided to the server system (2), and the server system (2) generates avirtual profile (212) of the preparation from the digital image data(31) at the highest available or desired magnification and in thehighest available or desired section of the preparation.
 37. The virtualmicroscope system as recited in claim 36, wherein the greatest availableor desired section of the preparation is a total view of thepreparation.
 38. The virtual microscope system as recited in claim 36,wherein the digital image data (31) provided by the image recordingsystem is reformatted in a compressed image format, in particularenhanced compressed wavelet format (ECW) via the first software (231)for generating virtual profiles (212), and the virtual profiles (212)are stored in the compressed image format in the case database (222).39. The virtual microscope system as recited in claim 36, wherein ineach case exactly one image region of the selected virtual profile (212)selected by the user (13) is requested from the case database (222) ofthe server system (2) at a compression level corresponding to theselected resolution or is loaded from a mobile data carrier anddisplayed.